Scialytic light navigation

ABSTRACT

A system and method for tracking an object within a surgical field are described. A system may include a lighting component to illuminate a surgical field, and a camera device to capture an image of a tracked device within the surgical field. The system may include a rotational component configured to rotate with respect to the lighting component. The camera device may couple to the rotational component to rotate with respect to the lighting component, such as in response to an obstruction of a tracked object being detected.

CLAIM OF PRIORITY

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/609,893, filed on Dec. 22, 2017, the benefit ofpriority of which is claimed hereby, and which is incorporated byreference herein in its entirety.

BACKGROUND

Cameras, object recognition technology, and tracking systems may beutilized to assist with surgical operations. This technology may be usedto locate surgical instruments or track the location of patientoperational points. Previous systems have relied on a static cameradevice. In the surgical field, object recognition and object trackingwith a static camera device proves difficult, such as when a surgeon,surgical assistant, robotic device, or other object blocks the staticcamera device from detecting a tracked object. When the tracked objectis not detected, aspects of a surgical procedure may need to stop,causing delays and initiation steps to be repeated. Some camera devicesare located on moving carts, but still must be static while trackingobjects during a procedure, and thus suffer from the same issues asother static cameras.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. The drawings illustrate generally, by way of example, butnot by way of limitation, various embodiments discussed in the presentdocument.

FIG. 1 illustrates a camera and light system in accordance with someembodiments.

FIG. 2 illustrates a surgical procedure setup including a camera andlight system in accordance with some embodiments.

FIG. 3 illustrates a multiple camera and light system in accordance withsome embodiments.

FIG. 4 illustrates a flowchart illustrating a technique for using acamera and light system in accordance with some embodiments.

FIG. 5 illustrates a flowchart for scialytic light navigation inaccordance with some embodiments.

FIG. 6 illustrates an example simulation of an overhead view of asurgical field in accordance with some embodiments.

FIG. 7 illustrates a block diagram of an example machine upon which anyone or more of the techniques discussed herein may perform in accordancewith some embodiments.

DETAILED DESCRIPTION

Systems and methods for using a camera device to track objects within asurgical field that is illuminated by a lighting device are describedherein. The systems and methods herein may include a camera devicecoupled to a lighting device, such as via an arm. The camera device maytrack objects using optical navigation. Optical navigation systems relyon an unobstructed line of sight between tracking elements and sensors(e.g., within the camera device) that receive tracking signals from thetracking elements. When the line of sight is obstructed (by a surgeon'sarm or a robotic arm for example), tracking signals being transmittedfrom the tracking element may not be received by the sensors or thecamera device. As a result, errors may occur in tracking the objects. Inan example where obstruction occurs, the optical navigation system maybe discontinued, the surgical procedure stopped, or an error message maybe conveyed to the surgeon. These changes may be temporary, such asuntil the line of sight returns or the system is reset.

The systems and methods herein describe a navigation system that quicklyidentifies line of sight issues so that obstructions may be resolvedwithout significant delay. The navigation systems and methods describedherein may improve a field of view or reduce possible errors associatedwith obstruction along a line of sight between a tracking element and asensor/camera device.

In an example, a system or method may track objects within a surgicalfield via a motorized navigation camera coupled to a scialytic light. Ascialytic light, as discussed herein may be used as an example type oflight for use within a surgical field, such as a light or plurality oflights configured to reduce or remove shadows within the surgical field.The motorized navigation camera may include a camera device, such as adepth camera, field of view camera, single camera, dual camera,plurality of cameras, or the like, which may be moveable, such as with amotor or by hand (e.g., with respect to a light). In another example,multiple cameras (e.g., camera devices, navigation cameras, depthcameras, etc.) may be used, such as using two or more camera devicescoupled to a light (e.g., on opposite sides of the light, or havingdifferent lines of sight).

In an example, a scialytic light may be coupled to a camera device, andboth may be communicatively coupled to a processor. The processor maycontrol the scialytic light or the camera (or a motor to rotate thecamera). When controlling the scialytic light, the processor may causethe scialytic light to display a special illumination mode (e.g., blink,change color, etc.) to indicate that there is a line of sight error withthe camera device. Alerting the surgeon using the special illuminationmode may allow the surgeon to avoid needing to look at a screen outsidethe surgical field.

A camera device used for tracking within a surgical field may comprise acamera, or a plurality of cameras communicatively coupled to a computerimage analysis system. By utilizing the camera mesh and the computerimage analysis system, objects involved in the surgical procedure may betracked, such as instruments (e.g., scalpel, implant insertion tools), abody part, or an implant. Tracking an object is not limited to a tool orinstrument used by the surgeon. The tracked object may include a bodypart, such as a leg or hand, the location of an incision, thepositioning of a bone inside the body part, or the location of an organwithin the patient. In an example, the camera device may track multipleobjects simultaneously.

The camera device may be configured in many ways for tracking an objector an optical tracker. The camera device may include a depth camera. Thecamera device may use infrared light to track an object or an opticaltracker. In this example, the camera device may be located at a fixedposition within a surgical field during a procedure (although, asdescribed below, the camera may rotate when an obstruction is detectedor predicted). In an example, the camera devices may includestereoscopic cameras (e.g., two cameras within a single housing).

The tracking data imaged by a camera device or collected by an imageprocessing system (e.g., using a processor) may be used to determine aposition and an orientation of a tracked object within a virtualthree-dimensional coordinate system (e.g., within the surgical field).The tracking data may include the position and direction of the cameradevice. The image processing system may output the position and theorientation of the tracked object. The output data may includecoordinates in a virtual three-dimensional coordinate system. The outputmay include one or more of the captured synchronized images thatincludes the tracked object. Using the tracking data, a processor maydetermine that a tracked object is obstructed or may predict that atracked object is going to be or likely to be obstructed in the nearfuture (e.g., within a second, within a few frames, etc.).

Robotics have become a useful tool for assisting the surgeon in thesurgical field. A robotic device may assist in the surgical fieldperforming tasks such as biopsies, electrode implantation for functionalprocedures (e.g., stimulation of the cerebral cortex, deep brainstimulation), open skull surgical procedures, endoscopic interventions,other “key-hole” procedures, arthroplasty procedures, such as total orpartial knee replacement, hip replacement, shoulder implant procedures,or the like. In an example, a surgical procedure may use a surgicalrobot. The surgical robot may include a tracker, which a tracking systemmay use to determine a relative location of the surgical robot within acoordinate system or a surgical field. The surgical robot may have adifferent coordinate system or tracking system (e.g., using knownmovements of the surgical robot to keep track of an end effector of arobotic arm of the surgical robot, which may include using sensors, suchas a gyroscope, magnetoscope, accelerometer, etc.). In an example, aprocessor may be used to coordinate or translate information from thesurgical robot coordinate or tracking system with a camera-basedtracking system.

An optical tracker may be attached to an object for tracking the objectin an optical tracking system, in accordance with some examples. Theoptical tracker may include one or multiple reflective components or anactive marker such as a light emitting diode (LED), which may be at aset position on the optical tracker (e.g., relative to one another). Theoptical tracker may be attached to an object in the surgical field suchas an instrument or a body part (e.g., a bone). The position andorientation of the object relative to the object tracker may be set andrecorded such that an optical tracking system may know the position andorientation of the object by determining the position and orientation ofthe optical tracker (e.g., using a camera device to capture an image orinformation about the optical tracker). The position and orientation ofthe optical tracker may be determined by detecting the one or multiplereflective components and correlating the detected reflective componentpositioning to the known relative positioning of the multiple reflectivecomponents on the optical tracker.

FIG. 1 illustrates a camera and light system 100 in accordance with someembodiments. The camera and light system 100 may be used in a surgicalfield for lighting, tracking objects, or image collection. The cameraand light system 100 includes a lighting component 106, which mayinclude a plurality of lights. The lighting component 106 may be coupledto a rotational component 108. The rotational component 108 is coupledto a camera attachment 110 in the example shown in FIG. 1. The cameraattachment 110 includes a camera device 112 at a distal end. In anotherexample, not shown, the camera device 112 may be coupled directly to therotational component 108. In either case, the camera device 112 iscoupled to the lighting component 106 via the rotational component 108,the rotational component 108 configured to rotate the camera device 112with respect to the lighting component 106. Rotating the camera device112 with respect to the lighting component 106 may include rotating thecamera device 112 without moving or rotating the lighting component 106.In an example, rotating the camera device 112 may include rotating inresponse to a parameter defining a line of sight (e.g., to a marker) orline of sight obstruction (e.g., a line of sight quality parameter). Therotational component 108 may be motorized, such that the rotationalcomponent 108 may be controlled to change the field of view 114 byrotating the camera device 112.

The camera and light system 100 may include a housing 104 for thelighting component 106, which may be coupled to a mounted arm 102. In anexample, the mounted arm 102 may in turn be coupled to a fixed portionof a surgical field (e.g., a ceiling of a surgical room). The cameradevice 112 may rotate with respect to the housing 104, the lightingcomponent 106, the mounted arm 102, or the fixed portion of the surgicalfield. In another example, the mounted arm 102 may be fixed to a mobilecart (which may be locked in a particular location during use).

The mounted arm 102 may be sufficiently rigid to prevent movement of thehousing 104 or the lighting component 106 during rotation of the cameradevice 112. In an example, the mounted arm 102 may include one or moreencoders (for position determination) in one or more joints of themounted arm 102 to determine the position of the housing 104, thelighting component 106, the rotational component 108, or the cameradevice 112. The housing 104 may include a handle for housingplacement/adjustment by the surgeon. When surgeon has placed thelighting component 106 in position (e.g., by using the handle),positions of the lighting component 106 and the camera device 112 may beknown by way of mounted ceiling arm joint 102 encoders and system design(e.g., known lengths or sizes of the components of the camera and lightsystem 100). The handle may have a button to active or shut off a brakein one or more joints of the mounted arm 102 (e.g., to stop the mountedarm 102 from moving or allow it to move). This button may permit themounted arm 102 to move when and where the surgeon wants and stop themounted arm 102 (by activating the brakes) to prevent any unintentionalmovements during surgery or movements caused by rotational movement ofthe camera device 112.

The camera device 112 may have a field of view 114, for example to trackobjects within a surgical field. The lighting component 106 may be usedto illuminate aspects of the surgical field, such as within the field ofview 114. The camera device 112 may track an object 116 within the fieldof view 114. A line of sight (e.g., a line within the field of view 114)from the camera device 112 to the tracked object 116 may becomeobstructed (e.g., partially or totally). In accordance with adetermination that the line of sight is obstructed or is going to becomeobstructed, the camera device 112 may rotate (changing the field of view114 and the line of sight) via the rotational component 108 to obtain aclear line of sight to the tracked object 116.

In an example, an obstruction may be detected for the tracked object 116when one or more aspects of the tracked object 116 are undetectable bythe camera device 112. For example, the tracked object 116 may have aplurality of reflective components for infrared detection by the cameradevice 112. When one or more of the plurality of reflective componentsor one or more active components, such as an LED, are undetectable bythe camera device 112, the camera device 112 may rotate to reobtain aline of sight that allows the camera device 112 to detect the pluralityof reflective components.

In an example, to predict that the line of sight is going to becomeobstructed, a processor (e.g., at a computing device, at a mobiledevice, within the camera and light system 100, within the camera device112, in a remote device, or the like) may determine from previous frames(two or more) that an object (e.g., an instrument, the surgeon, asurgical assistant, a robotic arm, etc.) is moving into a line of sightbetween the camera device 112 and the tracked object 116 or a portion ofthe tracked object 116. For example, movement of the object may bedetected in a direction to the line of sight from a first frame to asecond frame of the previous frames. In response to detecting themovement of the object towards the line of sight, the camera device 112may move to a new location such that a new line of sight from the cameradevice 112 to the tracked object 116 is not predicted to be obstructedby the object.

In another example, when the object that is going to obstruct thetracked object 116 is a robotic arm or other computer controlled device(i.e., an autonomous device that moves according to a controller), arobotic controller or other controller may identify that movement of theobject is going to obstruct the line of sight between the camera device112 and the tracked object 116. This information may be communicated tothe camera device 112 to instruct the camera device 112 to rotate beforethe tracked object 116 is obstructed. For example, a robotic arm may beinstructed by a robotic controller to move to a set of coordinates.Knowing the dimensions of the robotic arm, the previous location of therobotic arm and the coordinate, the robotic controller may determinewhether the robotic arm will intersect the line of sight (e.g., usinginformation from the camera device 112 about the location of the trackedobject 116 and the location of the camera device 112). In anotherexample, the robotic controller may send the coordinates to the cameradevice 112 or a computing device to determine whether the robotic armwill intersect the line of sight. The camera device 112 may outputcoordinates of a location of the tracked object 116 in a firstcoordinate system, as well as a location within the first coordinatesystem of the camera device 112 itself (e.g., at the origin, orincluding a rotational component). The robotic arm coordinates may beoutput in a second coordinate system. A processor may convert betweenthe first and second coordinate systems to determine whether the roboticarm will intersect with the line of sight. In an example, the cameradevice 112 may include a wired or wireless communication component tocommunicate coordinates, images, tracked information or the like to theprocessor or other device. A wired connection may connect via themounted arm 102. A wireless connection may include a transceiver, whichmay be located within the camera device 112 or the housing 104.

In an example, the lighting component 106 may be used to indicate to auser (e.g., a surgeon) that the line of sight between the camera device112 and the tracked object 116 is obstructed or for other errors. Theindication may include a flash of the lighting component 106, a changein light color, a pulse, or the like. The indication may cause a roboticarm to stop moving, move out of a line of sight or the field of view114, or the like.

The camera device 112 may use infrared (IR) light sensors with passivereflective spheres (e.g., on the tracked object 116) for tool, anatomy,or robotic tracking (e.g., position and orientation). In an example,other tracking technologies may be used, such as structured light ortime of flight to track the object 116, such as by creating a 3D model.In an example, the camera device 112 may move around the lightingcomponent 106 by via motorized link when the signal between the trackedobject 116 and the camera device 112 is lost, to look for and retrieve asignal from the tracked object 116 to avoid navigation interruption.

In an example, the camera device 112 may rotate in a single direction,moving as fast as possible to find the tracked object (e.g., via a motorwithin the rotational component 108). In another example, the cameradevice 112 may rotate in either direction (e.g., to a closest locationwithout an obstructed view). In yet another example, the camera device112 may rotate continuously, such that the camera device 112 always hasa clear line of sight to the tracked object 116 at least once perrotation. The rotational speed may be configured such that the cameraaccurately tracks the object 116 even with only one clear line of sight(e.g., for imaging or sensing) once per rotation. The rotational speedmay be aligned with a shutter speed or detection speed such that clearimages may be taken at predetermined or specified locations eachrotation. For example, when the camera device 112 takes 60 images persecond and the camera device 112 needs two images per second toaccurately track the object 116, the rotation speed may be set to causethe camera device 112 to rotate twice per second, giving 30 images perrotation, with at least two images not obstructed per second. The speedmay be varied based on the number of images taken per second and thenumber of clear line of sight images needed per second. In reality, thecamera device 112 is likely to have far more than one unobstructed lineof sight per rotation, so in these examples the camera device 112 mayobtain many more usable location images per rotation.

The camera mounting system (e.g., rotational component 108, cameraattachment 110, etc.) may be calibrated so that the physical position ofthe camera device 112 is known within a virtual coordinate system or thesurgical field at any rotational position. In some examples, therotational component 108 may include encoders or other rotationalposition sensors to provide an accurate rotational location of thecamera attachment 110 and camera device 112. With the ability toaccurate track rotational position, the exact field of view, such asfield of view 114, may be calculated at any point in the rotation of thecamera device 112. In some examples, additional fixed tracking markerscan be positioned though out the surgical field to further assist intransforming the virtual coordinate system between rotational positionsof camera device 112. The fixed tracking markers (not specifically shownin FIG. 1) may be mounted to a surgical table or other immovable objectswithin the surgical field. The fixed tracking markers may be utilized toaccount for any movement of the tracked object 116 between positionalmovements of the camera device 112. In other examples, movements of thecamera device 112 are sufficient fast to make any movement of thetracked object 116 negligible between frames. In yet other examples,tracked instruments and/or robotic arms can be utilized to assist intransforming the virtual coordinate system to the new camera position.For example, in scenarios utilizing a robotic arm, a portion of therobot can include tracking marker, which may be tracked by the cameradevice 112 as well as having a known fixed relationship to the roboticarm. The known fixed relationship to the robotic arm can be utilized tovalidate or calculate any interim movement of the tracked object 116during movement of the camera device 112 from one position to a secondposition.

The camera attachment 110 may be rigid or may be moveable (e.g.,adjustable in length, angle, or orientation to the lighting component106), but during operation it may be fixed. The camera attachment 110 orthe camera device 112 may be calibrated such that the field of view 114includes an entire surgical field or a desired portion of the surgicalfield. In an example, the moveable arm (e.g., the camera attachment 110)may include an anthropomorphic-like robotic arm having one or moreencoders in one or more joints to determine the position of the cameradevice 112 relative to the rotational component 108.

FIG. 2 illustrates a surgical procedure setup 200 including the cameraand light system 100 in accordance with some embodiments. The surgicalprocedure setup 200 illustrates a surgeon 202, a patient 204, a surgicalinstrument 206 (which may be an example of an object tracked by thecamera and light system 100), a robotic arm 208, a display device 210,and a computing device 212, each of which may be included or omitted ina given surgical procedure. In an example, the robotic arm 208 may betracked by the camera and light system 100. As shown in FIG. 2, thefield of view 114 may be less than an entire surgical field. The cameradevice 112 may rotate as described above, for example in response todetecting an obstruction or predicted obstruction of a line of sightfrom the camera device 112 to the surgical instrument 206 (or othertracked object), such as an obstruction by the surgeon 202, the roboticarm 208, or the like.

The display device 210 may be used to display information for thesurgical procedure, which may include tracking information from thecamera device 112. In an example, the display device may show an alert,such as when a line of sight to the surgical instrument 206, which maybe a tracked object, from the camera device 112 is obstructed. Thecomputing device 212 may be used to determine tracking information,compare or convert coordinates (e.g., coordinates of a location of aportion of the robotic arm within a first coordinate system andcoordinates of a location of the surgical instrument 206 within a secondcoordinate system).

The robotic arm 208 may be used by the surgeon 202 to perform a surgicalprocedure, such as on a knee joint of the patient 204. The robotic arm208 may use tracking information from the camera device 112, which maytrack a bone of the patient 204 (e.g., a femur or a tibia) or thesurgical instrument 206, or another device, tool, or aspect of patientanatomy to perform the surgical procedure.

As noted above, fixed tracking markers may, in some examples, be locatedthroughout the surgical field. In this example, the fixed trackingmarkers may be affixed to the surgical table supporting the patient 204.Further, fixed tracking markers may be affixed to portions of the robot208. In yet other examples, fixed tracking markers may be affixed toportions of the patient not directly involved in the surgery and notlikely to be moved.

FIG. 3 illustrates a multiple camera and light system 300 in accordancewith some embodiments. The multiple camera and light system 300 shown inFIG. 3 includes four camera devices 302A-302D, although two or threecamera devices may be used, or more than four camera devices may beused. The multiple camera and light system 300 uses a plurality ofcamera devices 302A-302D for a plurality of field of views 306A-306D. Inan example, the multiple camera and light system 300 may includecomponents similar to those of FIG. 1, including a housing 104, alighting component 106, a rotational component 108, or a support arm102. The camera devices 302A-302D may be connected to the housing 104 orthe lighting component 106 via camera attachments 304A-304D,respectively.

In an example, the multiple camera and light system 300 may rotate thecamera devices 302A-302D, such as in response to an obstruction detectedor predicted in a line of sight to a tracked object. In an example, themultiple camera and light system 300 may track one or multiple objects.Based on the number of objects tracked, different techniques may be usedto adjust rotation of the camera devices 302A-302D. In an example, whena single object is tracked, the multiple camera and light system 300 mayrotate only when all respective lines of sight from the camera devices302A-302D to the single tracked object are obstructed or predicted to beobstructed, when a plurality of lines of sight are obstructed to asingle tracked device, or when at least one line of sight is obstructed.For example, the camera devices 302A-302D may be rotated in differenttechniques when only one line of sight is obstructed, or in otherexamples, when two lines of sight, three lines of sight, or all fourlines of sight are obstructed. Similarly, when two, three, or more thanfour camera devices are used, techniques may be used to rotate thecamera devices when one, two, three, four, or more lines of sights areobstructed. In an example, the multiple camera and light system 300 maydecrease the number of possibilities of line of sight issues by coveringthe largest field of view as possible (e.g., 360° in a best case). Evenin this case, rotational movement may occur when all lines of sight areobstructed to retrieve at least one line of sight permitting tracking.

In another example, a lead camera device 302A may be designated when asingle object is tracked. In this example, when a line of sight isobstructed or predicted to be obstructed, tracking may be switched toone of the backup camera devices 302B-D, such as depending on which ofthe backup camera devices 302B-D have a best line of sight (e.g., leastobstructed, and may include a specified order when no obstructions arepresent or predicted). The camera devices 302A-302D may be rotated suchthat the lead camera device 302A regains a line of sight to the singletracked object, or the backup camera 302B with a clear line of sight maybe designated a new lead camera device. In this example, the backupcamera devices 302B-D may operate with different settings, such as lowerpower, taking fewer images or sensor collections per time period, lowerresolution for image capture, taking images or collecting sensor data,but not sending it or not processing it, or the like. Once activated tolead camera, the settings may be restored to ideal operating properties.

In an example, more than one object may be tracked within a surgicalfield. When tracking more than one object, the camera devices 302A-302Dmay be rotated when any line of sight to any of the tracked objects isobstructed or predicted to be obstructed. For example, tracking threeobjects in FIG. 3 would have twelve sight lines (one from each cameradevice to each tracked object), and when any of the twelve areobstructed, the camera devices 302A-302D are rotated.

In another example, the camera devices 302A-302D may be rotated when alllines of sight to any of the tracked objects are obstructed or predictedto be obstructed. For example, tracking three objects in FIG. 3 wouldhave twelve sight lines, and when all four sight lines to a singletracked object are obstructed, the camera devices 302A-302D are rotated.Thus in this example, multiple sight lines (up to three per trackedobject) may be obstructed without rotation, so long as at least onesight line per tracked object is not obstructed. In this example, notall tracked objects may be visible to all camera devices or to even asingle camera device, but each tracked object is visible to the multiplecamera and light system 300 because at least one line of sight to thetracked object from at least one of the camera devices is clear.

In yet another example, the camera devices 302A-302D may be rotated whenmore than one line of sight from one of the camera devices 302A-302D toa tracked object are obstructed or predicted to be obstructed. Forexample, when two or three lines of sight to an object are obstructed,the camera devices 302A-302D may be rotated. The examples related toFIG. 3 have been presented with three tracked objects and four cameradevices, but they may be applied to two tracked objects, or more thanthree tracked objects, two or three camera devices, or more than fourcamera devices.

FIG. 4 illustrates a flowchart illustrating a technique 400 for using acamera and light system in accordance with some embodiments. Thetechnique 400 includes an optional operation 402 to receive, for examplefrom a camera device, first image information of the surgical field.

The technique 400 includes a decision operation 404 to analyze the firstimage information. Decision operation 404 may include determiningwhether a sight line from the camera device to a tracked object isobstructed. Optionally, the decision operation 404 may includepredicting that a sight line is going to become obstructed bycalculating a trajectory of an object likely to obstruct the sight linefrom previous images or from information received about futurecoordinates of a robotic arm. In response to determining that the sightline is at least partially obstructed, the technique 400 may proceed toan operation 406. Operation 406 includes causing the camera device torotate about a lighting component. After rotating the camera device, thetechnique may continue to operation 408. In another example, such as inresponse to determining that the sight line is not obstructed, thetechnique 400 may continue to operation 408. Rotating the camera devicemay include rotating the camera device to a location where the trackedobject is not obstructed in a second sight line to the camera devicewithout the lighting component rotating, for example using an externaldevice coupled to the system or another camera coupled to the system todetermine the location where the tracked object is not obstructed. In anexample, rotating the camera device includes rotating the camera devicein response to determining that the tracked object is at least partiallyobstructed from the sight line of the camera device and a second sightline of a second camera device. In another example, the camera devicemay be rotated by activating a motor to cause a rotational componentcoupled to the light component (e.g., a proximal end of the rotationalcomponent is affixed to the lighting component and a distal end of therotational component is affixed to the camera device) to rotate. In yetanother example, the camera device may be constantly rotated about thelighting component.

The technique 400 includes an operation 408 to receive second imageinformation (e.g., from the camera device) of the surgical fieldindicative of a position and orientation of the tracked object withinthe surgical field. In an example, the position and orientation of thetracked object may be determined from the first image information. In anexample, operation 408 may use a parameter defining quality of the lineof sight (e.g., degree of obstruction) to determine whether rotation isneeded.

The technique 400 includes an operation 410 to output the position andthe orientation of the tracked object. The technique 400 may includeflashing a light of the lighting component in response to detecting theat least partial obstruction. In an example, the flash or indicatingportion of the lighting component may be performed with visualwavelengths, while the camera device may be adapted to capturenon-visible wavelengths (e.g., infrared). The visible flash alerting thesurgeon and the non-visible light used by the camera devices allows acamera and light system using the technique 400 to include lightingbased signaling for the surgeon while not impacting the ability of anyof the camera devices to detect and track objects.

The lighting component may include a plurality of lights, the pluralityof lights fixed with respect to the surgical field. The plurality oflights may be scialytic lights. The camera device may be an infrareddepth camera, which may capture infrared images or image information(e.g., from reflectors on a tracked object). The lighting component maybe secured within a housing, and a ceiling mounted arm may be connectedto the housing or a mobile cart.

FIG. 5 illustrates a flowchart 500 for scialytic light navigation inaccordance with some embodiments. The flowchart 500 illustratesiteratively analyzing images captured by a camera to determine whether aline of sight (LS) to a tracked object is obstructed or not obstructed.In response to determining the LS is obstructed, the flowchart 500includes causing the camera to rotate about the lighting component. Inresponse to determining that the LS is not obstructed, the flowchart 500includes retaining the camera at its position, and outputting theposition or orientation of the tracked object. The flowchart 500 mayoutput a parameter defining LS quality (e.g., an amount the LS isobstructed, an indication of whether or to what degree the LS is like tobe obstructed, or the like).

In the example shown in flowchart 500, the camera takes a first image.This image is analyzed to determine if the LS is obstructed or not. Ifthe LS is not obstructed, the flowchart 500 includes outputting theposition and orientation of the tracked object, as well as optionally aparameter defining LS quality. If the LS is obstructed, the camera mayrotate (the amplitude may correspond to the parameter definingobstruction). After rotating, the camera takes another image. This imageis then analyzed to determine if line of sight is still obstructed. IfLS is still obstructed, the camera moves and the operations may berepeated until the LS is not obstructed. If the LS is not obstructed thesystem outputs position and orientation of the tracked object and theparameter defining LS quality.

FIG. 6 illustrates an example simulation 600 of an overhead view of asurgical field in accordance with some embodiments. The simulation 600is a visual example that may be used in determining an initial line ofsight (or example lines of sight). The simulation 600 may include atable 602, an operating arm 604, a robot 606, a camera 608, and one ormore people (e.g., a surgeon 610, a resident 612, an anesthesiologist614, or a nurse 616) or other objects (e.g., a display screen 618). Inan example, a user (e.g., a surgeon) may arrange the simulation 600 tolook like the user's surgical field. The components of the simulation600 may be moved or arranged by the user or may be placed automaticallyby a processor running the simulation 600. In an example, objects (e.g.,the table 602) may be static and other objects (e.g., the robot 606 orthe surgeon 610) may be moveable.

The simulation 600 may include a tracked object 620 (e.g., passive oractive) for determining a line of sight to the camera 608. Thesimulation 600 may try different lines of sight corresponding todifferent rotations of the camera 608 (e.g., around a lighting componentas discussed above) to determine a best or initial line of sight. Thesimulation 600 may cause objects that are moveable to move andrecalculate a best or initial line of sight. In an example, thesimulation 600 may determine one or more locations or orientations ofvarious objects within the simulation 600 that cause the camera 608 tohave fewer or more lines of sight to the tracked object 620. Thearrangements with fewer lines of sight may be identified to the user sothat these may be avoided during an actual surgical procedure. Forexample, the simulation 600 may move the robot 606 to where the resident612 is in FIG. 6, and determine that this leads to fewer lines of sight(or an increase in lost line of sight). The position or orientation ofthe robot 606 may then be output, such as to a display screen to warnthe user to avoid placing the robot in the position where the resident612 is in FIG. 6.

In another example, at the beginning of a surgical procedure, a cameramay rotate 360° around the lighting component to find the positionrelative to the best line of sight (e.g., without needing the simulation600 or to verify or validate findings of the simulation 600). In thisexample, the best line of sight may be determined by using a parameterdefining line of sight such as tracked object volume or amount of lightreceived. To do this the surgeon may manually position an object to betracked above the surgical table/field and trigger a calibrationprocedure. During the surgical procedure the camera may always belooking for the best line of sight. The best line of sight may bedefined as a line of sight that provides the most accurate tracking.

FIG. 7 illustrates a block diagram of an example machine 700 upon whichany one or more of the techniques discussed herein may perform inaccordance with some embodiments. In alternative embodiments, themachine 700 may operate as a standalone device or may be connected(e.g., networked) to other machines. In a networked deployment, themachine 700 may operate in the capacity of a server machine, a clientmachine, or both in server-client network environments. In an example,the machine 700 may act as a peer machine in peer-to-peer (P2P) (orother distributed) network environment. The machine 700 may be apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a mobile telephone, a web appliance, a networkrouter, switch or bridge, or any machine capable of executinginstructions (sequential or otherwise) that specify actions to be takenby that machine. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methodologiesdiscussed herein, such as cloud computing, software as a service (SaaS),other computer cluster configurations.

Machine (e.g., computer system) 700 may include a hardware processor 702(e.g., a central processing unit (CPU), a graphics processing unit(GPU), a hardware processor core, or any combination thereof), a mainmemory 704 and a static memory 706, some or all of which may communicatewith each other via an interlink (e.g., bus) 708. The machine 700 mayfurther include a display unit 710, an alphanumeric input device 712(e.g., a keyboard), and a user interface (UI) navigation device 714(e.g., a mouse). In an example, the display unit 710, input device 712and UI navigation device 714 may be a touch screen display. The machine700 may additionally include a storage device (e.g., drive unit) 716, asignal generation device 718 (e.g., a speaker), a network interfacedevice 720, and one or more sensors 721, such as a global positioningsystem (GPS) sensor, compass, accelerometer, or other sensor. Themachine 700 may include an output controller 728, such as a serial(e.g., Universal Serial Bus (USB), parallel, or other wired or wireless(e.g., infrared (IR), near field communication (NFC), etc.) connectionto communicate or control one or more peripheral devices (e.g., aprinter, card reader, etc.).

The storage device 716 may include a machine readable medium 722 onwhich is stored one or more sets of data structures or instructions 724(e.g., software) embodying or utilized by any one or more of thetechniques or functions described herein. The instructions 724 may alsoreside, completely or at least partially, within the main memory 704,within static memory 706, or within the hardware processor 702 duringexecution thereof by the machine 700. In an example, one or anycombination of the hardware processor 702, the main memory 704, thestatic memory 706, or the storage device 716 may constitute machinereadable media.

While the machine readable medium 722 is illustrated as a single medium,the term “machine readable medium” may include a single medium ormultiple media (e.g., a centralized or distributed database, and/orassociated caches and servers) configured to store the one or moreinstructions 724. The term “machine readable medium” may include anymedium that is capable of storing, encoding, or carrying instructionsfor execution by the machine 700 and that cause the machine 700 toperform any one or more of the techniques of the present disclosure, orthat is capable of storing, encoding or carrying data structures used byor associated with such instructions. Non-limiting machine readablemedium examples may include solid-state memories, and optical andmagnetic media.

The instructions 724 may further be transmitted or received over acommunications network 726 using a transmission medium via the networkinterface device 720 utilizing any one of a number of transfer protocols(e.g., frame relay, internet protocol (IP), transmission controlprotocol (TCP), user datagram protocol (UDP), hypertext transferprotocol (HTTP), etc.). Example communication networks may include alocal area network (LAN), a wide area network (WAN), a packet datanetwork (e.g., the Internet), mobile telephone networks (e.g., cellularnetworks), Plain Old Telephone (POTS) networks, and wireless datanetworks (e.g., Institute of Electrical and Electronics Engineers (IEEE)802.11 family of standards known as Wi-Fi®, IEEE 802.16 family ofstandards known as WiMax®), IEEE 802.15.4 family of standards,peer-to-peer (P2P) networks, among others. In an example, the networkinterface device 720 may include one or more physical jacks (e.g.,Ethernet, coaxial, or phone jacks) or one or more antennas to connect tothe communications network 726. In an example, the network interfacedevice 720 may include a plurality of antennas to wirelessly communicateusing at least one of single-input multiple-output (SIMO),multiple-input multiple-output (MIMO), or multiple-input single-output(MISO) techniques. The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding orcarrying instructions for execution by the machine 700, and includesdigital or analog communications signals or other intangible medium tofacilitate communication of such software.

Various Notes & Examples

Each of these non-limiting examples may stand on its own, or may becombined in various permutations or combinations with one or more of theother examples.

Example 1 is a system for tracking an object within a surgical field,the system comprising: a lighting component to illuminate a surgicalfield; a camera device coupled to the lighting component via arotational component configured to rotate the camera device with respectto the lighting component; a computing system communicatively coupled tothe camera device, the computing system including a processor and amemory device, the memory device including instructions that, whenexecuted by the processor, cause the computing system to: receive, fromthe camera device, first image information of the surgical field;analyze the first image information to detect that a sight line from thecamera device to a tracked object is at least partially obstructed;control the rotational component to rotate the camera device about thelighting component; receive, from the camera device subsequent to thecamera device rotating, second image information of the surgical fieldindicative of a position and orientation of the tracked object withinthe surgical field; and output the position and the orientation of thetracked object.

In Example 2, the subject matter of Example 1 includes, wherein therotational component is configured to rotate the camera device to alocation where the tracked object is not obstructed in a second sightline to the camera device without the lighting component rotating.

In Example 3, the subject matter of Examples 1-2 includes, wherein thelighting component includes a plurality of lights, the plurality oflights fixed with respect to the surgical field.

In Example 4, the subject matter of Example 3 includes, wherein theplurality of lights are scialytic lights.

In Example 5, the subject matter of Examples 1-4 includes, a secondcamera device coupled to the lighting component via the rotationalcomponent.

In Example 6, the subject matter of Example 5 includes, wherein thecomputing device is to control the rotational component to rotate thecamera device about the lighting component in response to determiningthat the tracked object is at least partially obstructed from the sightline of the camera device and a second sight line of the second cameradevice.

In Example 7, the subject matter of Examples 1-6 includes, wherein thecamera device is an infrared depth camera.

In Example 8, the subject matter of Examples 1-7 includes, wherein thesystem further includes a motor to cause the rotational component torotate.

In Example 9, the subject matter of Examples 1-8 includes, wherein thesystem further includes a housing or a mobile cart, including thelighting component, and a ceiling mounted arm connected to the housingor mounted arm connected to the mobile cart.

In Example 10, the subject matter of Examples 1-9 includes, wherein thecomputing device is further to cause a light of the lighting componentto flash in response to detecting the at least partial obstruction.

In Example 11, the subject matter of Examples 1-10 includes, wherein tocontrol the rotational component to rotate the camera device about thelighting component, the computing device is to control the rotationalcomponent to constantly rotate the camera device.

Example 12 is a lighting fixture comprising: a lighting component toilluminate a surgical field; a rotational component configured to rotatewith respect to the lighting component; a camera device configured to:couple to the rotational component to rotate with respect to thelighting component; and capture an image of a tracked device within thesurgical field.

Example 13 is a computer implemented method for tracking an objectwithin a surgical field, the method comprising: receiving, at aprocessor from a camera device, first image information of the surgicalfield; analyzing, using the processor, the first image information todetect that a sight line from the camera device to a tracked object isat least partially obstructed; communicating a signal to a rotationalcomponent associated with the camera device, the signal to rotate thecamera device about a lighting component; receiving, from the cameradevice subsequent to the camera device rotating, second imageinformation of the surgical field indicative of a position andorientation of the tracked object within the surgical field; andoutputting, from the processor, the position and the orientation of thetracked object.

In Example 14, the subject matter of Example 13 includes, whereincommunicating the signal to rotate the camera device includescalculating a position to rotate the camera device where the trackedobject is not obstructed in a second sight line to the camera devicewithout a lighting component rotating.

In Example 15, the subject matter of Examples 13-14 includes,communicating the signal to rotate the camera device includescalculating a position to rotate the camera device in response todetermining that the tracked object is at least partially obstructedfrom the sight line of the camera device and a second sight line of asecond camera device.

In Example 16, the subject matter of Examples 13-15 includes, whereincommunicating the signal to rotate the camera device includes causing,by the processor, activation of a motor to cause a rotational componentcoupled to a lighting component to rotate.

In Example 17, the subject matter of Examples 13-16 includes, generatingan alert signal to flash a light of a lighting component in response todetecting the at least partial obstruction.

In Example 18, the subject matter of Examples 13-17 includes, whereincommunicating the signal to rotate the camera device about a lightingcomponent, includes communicating the signal to constantly rotate thecamera device.

Example 19 is at least one non-transitory machine-readable mediumincluding instructions for tracking an object within a surgical field,which when executed by a processor, cause the processor to: receive,from a camera device, first image information of the surgical field;analyze the first image information to detect that a sight line from thecamera device to a tracked object is at least partially obstructed;communicating a signal to a rotational component associated with thecamera device, the signal to cause the camera device to rotate about alighting component; receive, from the camera device subsequent to thecamera device rotating, second image information of the surgical fieldindicative of a position and orientation of the tracked object withinthe surgical field; and output the position and the orientation of thetracked object.

In Example 20, the subject matter of Example 19 includes, wherein tocommunicate the signal to rotate the camera device, the instructionsfurther cause the processor to calculate a position to rotate the cameradevice where the tracked object is not obstructed in a second sight lineto the camera device without a lighting component rotating.

In Example 21, the subject matter of Examples 19-20 includes, wherein tocommunicate the signal to rotate the camera device, the instructionsfurther cause the processor to calculate a position to rotate the cameradevice in response to determining that the tracked object is at leastpartially obstructed from the sight line of the camera device and asecond sight line of a second camera device.

In Example 22, the subject matter of Examples 19-21 includes, wherein tocommunicate the signal to rotate the camera device, the instructionsfurther cause the processor to cause activation of a motor to cause arotational component coupled to a lighting component to rotate.

In Example 23, the subject matter of Examples 19-22 includes, whereinthe instructions further cause the processor to generate an alert signalto flash a light of a lighting component in response to detecting the atleast partial obstruction.

In Example 24, the subject matter of Examples 19-23 includes, wherein tocommunicate the signal to rotate the camera device about a lightingcomponent, the instructions further cause the processor to communicatethe signal to constantly rotate the camera device.

Example 25 is at least one machine-readable medium includinginstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform operations to implement of any ofExamples 1-24.

Example 26 is an apparatus comprising means to implement of any ofExamples 1-24.

Example 27 is a system to implement of any of Examples 1-24.

Example 28 is a method to implement of any of Examples 1-24.

Method examples described herein may be machine or computer-implementedat least in part. Some examples may include a computer-readable mediumor machine-readable medium encoded with instructions operable toconfigure an electronic device to perform methods as described in theabove examples. An implementation of such methods may include code, suchas microcode, assembly language code, a higher-level language code, orthe like. Such code may include computer readable instructions forperforming various methods. The code may form portions of computerprogram products. Further, in an example, the code may be tangiblystored on one or more volatile, non-transitory, or non-volatile tangiblecomputer-readable media, such as during execution or at other times.Examples of these tangible computer-readable media may include, but arenot limited to, hard disks, removable magnetic disks, removable opticaldisks (e.g., compact disks and digital video disks), magnetic cassettes,memory cards or sticks, random access memories (RAMs), read onlymemories (ROMs), and the like.

What is claimed is:
 1. A system for tracking an object within a surgicalfield, the system comprising: a lighting component to illuminate asurgical field; a camera device coupled to the lighting component via arotational component configured to rotate the camera device with respectto the lighting component; a computing system communicatively coupled tothe camera device, the computing system including a processor and amemory device, the memory device including instructions that, whenexecuted by the processor, cause the computing system to: receive, fromthe camera device, first image information of the surgical field;analyze the first image information to detect that a sight line from thecamera device to a tracked object is at least partially obstructed;control the rotational component to rotate the camera device about thelighting component; receive, from the camera device subsequent to thecamera device rotating, second image information of the surgical fieldindicative of a position and orientation of the tracked object withinthe surgical field; and output the position and the orientation of thetracked object.
 2. The system of claim 1, wherein the rotationalcomponent is configured to rotate the camera device to a location wherethe tracked object is not obstructed in a second sight line to thecamera device without the lighting component rotating.
 3. The system ofclaim 1, wherein the lighting component includes a plurality of lights,the plurality of lights fixed with respect to the surgical field.
 4. Thesystem of claim 3, wherein the plurality of lights are scialytic lights.5. The system of claim 1, further comprising a second camera devicecoupled to the lighting component via the rotational component.
 6. Thesystem of claim 5, wherein the computing device is to control therotational component to rotate the camera device about the lightingcomponent in response to determining that the tracked object is at leastpartially obstructed from the sight line of the camera device and asecond sight line of the second camera device.
 7. The system of claim 1,wherein the camera device is an infrared depth camera.
 8. The system ofclaim 1, wherein the system further includes a motor to cause therotational component to rotate.
 9. The system of claim 1, wherein thesystem further includes a housing, including the lighting component, anda ceiling mounted arm connected to the housing.
 10. The system of claim1, wherein the computing device is further to cause a light of thelighting component to flash in response to detecting the at leastpartial obstruction.
 11. The system of claim 1, wherein to control therotational component to rotate the camera device about the lightingcomponent, the computing device is to control the rotational componentto constantly rotate the camera device.
 12. A computer implementedmethod for tracking an object within a surgical field, the methodcomprising: receiving, at a processor from a camera device, first imageinformation of the surgical field; analyzing, using the processor, thefirst image information to detect that a sight line from the cameradevice to a tracked object is at least partially obstructed;communicating a signal to a rotational component associated with thecamera device, the signal to rotate the camera device about a lightingcomponent; receiving, from the camera device subsequent to the cameradevice rotating, second image information of the surgical fieldindicative of a position and orientation of the tracked object withinthe surgical field; and outputting, from the processor, the position andthe orientation of the tracked object.
 13. The method of claim 12,wherein communicating the signal to rotate the camera device includescalculating a position to rotate the camera device where the trackedobject is not obstructed in a second sight line to the camera devicewithout a lighting component rotating.
 14. The method of claim 12,wherein communicating the signal to rotate the camera device includescalculating a position to rotate the camera device in response todetermining that the tracked object is at least partially obstructedfrom the sight line of the camera device and a second sight line of asecond camera device.
 15. The method of claim 12, wherein communicatingthe signal to rotate the camera device includes causing, by theprocessor, activation of a motor to cause a rotational component coupledto a lighting component to rotate.
 16. The method of claim 12, furthergenerating an alert signal to flash a light of a lighting component inresponse to detecting the at least partial obstruction.
 17. The methodof claim 12, wherein communicating the signal to rotate the cameradevice about a lighting component, includes communicating the signal toconstantly rotate the camera device.
 18. At least one non-transitorymachine-readable medium including instructions for tracking an objectwithin a surgical field, which when executed by a processor, cause theprocessor to: receive, from a camera device, first image information ofthe surgical field; analyze the first image information to detect that asight line from the camera device to a tracked object is at leastpartially obstructed; communicating a signal to a rotational componentassociated with the camera device, the signal to cause the camera deviceto rotate about a lighting component; receive, from the camera devicesubsequent to the camera device rotating, second image information ofthe surgical field indicative of a position and orientation of thetracked object within the surgical field; and output the position andthe orientation of the tracked object.
 19. The at least onemachine-readable medium of claim 18, wherein to communicate the signalto rotate the camera device, the instructions further cause theprocessor to calculate a position to rotate the camera device where thetracked object is not obstructed in a second sight line to the cameradevice without a lighting component rotating.
 20. The at least onemachine-readable medium of claim 18, wherein to communicate the signalto rotate the camera device, the instructions further cause theprocessor to calculate a position to rotate the camera device inresponse to determining that the tracked object is at least partiallyobstructed from the sight line of the camera device and a second sightline of a second camera device.